Page views for proxy servers

ABSTRACT

Distinctions between resources explicitly selected by a user and resources indirectly selected may be enabled by identifying an explicitly selected web page or other resource as such in the request for the web page or other resource, which may allow the web page or resource to be differentiated from web pages or other resources that are requested as a consequence of their indirect selection. Moreover, a log of web pages or other resources explicitly selected by a user may be maintained at the client and later reference by a local processor or communicated to a host process seeking to differentiate directly and indirectly selected web pages or other resources. These techniques also may allow a proxy or other server to perform processing related to parentally controlled accounts or related to accurately tracking frequently requested resources such as web pages.

CLAIM OF PRIORITY

This application is a continuation of U.S. patent application Ser. No.13/243,828, filed on Sep. 9, 2011 (now allowed), which is a continuationof U.S. patent application Ser. No. 11/321,043, filed on Dec. 30, 2005(now U.S. Pat. No. 8,028,059), which is a continuation-in-part of U.S.patent application Ser. No. 10/858,394, filed on Jun. 2, 2004, which isa continuation-in-part of U.S. patent application Ser. No. 10/747,265,filed on Dec. 30, 2003, which claims priority to U.S. Provisional patentapplication Ser. No. 60/474,623, filed on Jun. 2, 2003. All of theseapplications are hereby incorporated by reference.

TECHNICAL FIELD

This description relates to proxy server processing.

BACKGROUND

Client computers can communicate with a server to remotely accessinformation stored at the server. The transfer of information betweenthe server and client computers may be provided by observing standardprotocols and using software applications. For example, a hypertextmarkup language (HTML) browser application at a client computer cancommunicate over the public Internet using standard communicationprotocols, such as Transfer Control Protocol/Internet Protocol (TCP/IP)and hypertext transfer protocols (HTTP), to retrieve resources such asweb pages or objects from a HTTP server. Web pages may include formattedtext as well as objects, such as multimedia elements including, forexample, embedded graphics, sounds, and/or video. Exemplary browserapplications include Netscape Navigator and Microsoft Internet Explorer.

The America Online (AOL) client is an example of a proprietary browserapplication. The AOL client executes on a client computer connected tothe AOL network. To receive web pages from servers on the AOL network,the AOL client communicates with servers on the AOL network usingproprietary communication protocols. When the AOL client seeks toreceive web pages available on HTTP servers located on the publicInternet, the AOL client communicates with a proxy server on the AOLnetwork. The proxy server then communicates with HTTP servers on thepublic Internet using standard communication protocols, such as TCP/IPand HTTP, to receive the web pages from the HTTP servers. The web pagesreceived from HTTP servers may be written in a standard language such asHTML, while the AOL client renders web pages written in a proprietarylanguage. Thus, once received, the proxy server translates the webpages, as necessary, into the proprietary language and forwards thetranslated web pages to the AOL client.

The HTTP specification provides for an HTTP header known as the RefererHeader that is intended to reveal the identity of a resource used tocall each object included on a displayed web page. More precisely, theHTTP Referer Header specifies “the address [a Uniform ResourceIdentifier](URI) of the resource from which the Request-URI wasobtained.” See “HyperText Transfer Protocol—HTTP/1.1,” RFC 2616,http://wwwv.w3.org/Protocols/rfc2616/rfc2616.html. An example of theHTTP Referer Header is shown during the processes undertaken when a usernavigates to www.aol.com. The rendering of the aol.com web page causesthe browser to obtain a number of other objects such as graphics. Foreach of the items being retrieved the HTTP Referrer Header indicates areferring page. For instance, a trace shows the following during therendering of one of the objects embedded within www.aol.com:

Uniform Resource Locator

(URL):http://m2.doubleclick.net/viewad/698862/roommates180x75.gif

Headers:

Referer:http://ad.doubleclick.net/adi/N2885.AOLcom/B910031.25;sz=180x75;ord=[timestamp]?

Accept-Language: en-us

Interestingly, even though it is the rendering of the aol.com webpagethat is initially triggering retrieval of the embedded object, thereferrer header identifies a resource other then aol.com. Specifically,the HTTP Referer Header indentifies“ad.doubleclick.net/adi/N2885.AOLcom/B910031.25;sz=180x75;ord=[timestamp]?”as the referring resource.

Generally, the failure of the HTTP Referer Header to identify aol.comresults from multiple levels of indirection or redirection needed toultimately load the embedded objects and the design of the HTTP RefererHeader to identity the “lowest” level of redirection rather then the“highest” level of redirection. More particularly, the HTTPspecification so requires the HTTP Referer Header to indicate theresource from which the requested URL was directly obtained. In keepingwith the earlier example, the above trace reveals that the followingdestination was called prior to calling the embedded object, likelybecause of an URL embedded within aol.com that specified thisdestination and resulted on the following request:

URL:http://ad.doubleclick.net/adi/N2885.AOLcom/B910031.25;sz=180x75;ord-[timestamp]?

Headers:

Referer: http://www.aol.com/

Accept-Language: en-us

As can be seen, the resource identified in the HTTP referrer header forthe request initially shown above (i.e., http://ad.doubleclick.net . . .) was retrieved before that first request was sent. And, that resourceredirected or otherwise instructed the browser to retrieve the resourceat http://m2.doubleclick.net/viewad/698862/roommates180x75.gif.Therefore, the http://ad.doubleclick.net . . . resource is the resourcefrom which the embedded http://m2.doubleclicknet . . . resource wasobtained, causing the HTTP Referer Header to identifyhttp://ad.doubleclick.net . . . rather than www.aol.com, which is theURL for the web page whose rendering triggered the embedded resource tobe retrieved.

Notably, implementations of the HTTP referrer handles frame pages byincluding the URL of the web page loaded into a frame, rather than theURL of the frame page.

SUMMARY

In one aspect, a detection is made as to whether a user has explicitlyrequested a first resource, and a request for a first resource isgenerated. When a user has explicitly requested the first resource, anindication is included in the request. The indication indicates thatthat the user explicitly requested the first resource. The request issent to a server computer and the first resource is received from theserver computer. The first resource is rendered, which includesgenerating a request for a second resource and sending the request forthe second resource. The request for the second resource does notinclude an indication that the user explicitly requested the secondresource; and

Implementations of this aspect may include one or more of the followingfeatures. For example, the server computer may be a proxy server. Arequest for a third resource may so be generated, and the request forthe third resource may include an indication that the user explicitlyrequested the third resource. The third request may be sent to a servercomputer, and the third resource may be received and rendered.

The detection may include detecting that the user has clicked on ahyperlink or that the user has typed a location of the first resourceinto a navigation bar of a browser. The first resource may be a web pageand the request for the first resource may be an HTTP request. Theindication may be a header such that generating a request for the firstresource includes inserting a header into the request for the firstresource. The header may be an HTTP header.

In another aspect, a request is received from a client system for afirst resource requested by a user. The request includes an indicationthat the user explicitly requested the first resource. The firstresource is sent to the client system. A request for a second resourceis received while the client system is rendering the first resource. Therequest for the second resource does not include an indication that theuser explicitly requested the second resource. The second resource issent to the client system.

Implementations of this aspect may include one or more of the followingfeatures. For example, processing related to access controls may beperformed based on the indication that the user explicitly requested thefirst resource included in the request for the first resource. Theaccess controls may relate to a parental control service. The processingmay included including an identification of the first resource in areport to a parental account because the request for the first resourceincludes the indication that the user explicitly requested the firstresource. The processing additionally, or alternatively, may includeexcluding an identifier for the second resource in the report to theparental account because the request for the second resource does notinclude an indication that the user explicitly requested the secondresource.

Additionally, or alternatively, processing related to trackingfrequently requested resources may be performed based on the indicationthat the user explicitly requested the first resource included in therequest for the first resource. The processing may include including thefirst resource in a count of requested resources because the request forthe first resource includes the indication that the user explicitlyrequested the first resource. Additionally, or alternatively, theprocessing may include excluding the second resource from the count ofrequested resources because the request for the second resource does notinclude the indication that the user explicitly requested the secondresource.

The first resource is a web page and the request for the first resourceis an HTTP request. The indication that the user explicitly requestedthe first resource may include a header such that generating a requestfor the first resource includes inserting a header into the request forthe first resource. The header may be an HTTP header.

In another aspect, a request is received from a client system for afirst resource explicitly requested by a user of the client system andthe first resource is sent to the client system. A communication isreceived from the client system separately from the request for thefirst resource. The communication identifies the first resource as aresource explicitly request by the user of the client system. Processingrelated to at least one of access controls or tracking frequentlyrequested resources is performed based on the communication.

Implementations of this aspect may include one or more of the followingfeatures. For example, The first resource may be a web page and therequest for the first resource may be an HTTP request. Processingrelated to access controls may include including an identification ofthe first resource in a report to a parental account because thecommunication identified the first resource as a resource explicitlyrequest by the user of the client system. Additionally, oralternatively, processing related to access controls may includeexcluding an identifier for a second resource in the report to theparental account if a communication identifying the second resource as aresource explicitly requested by the user of the client system was notreceived from the client system.

Processing related to tracking frequently requested resources mayinclude including the first resource in a count of requested resourcesbecause the communication identified the first resource as a resourceexplicitly request by the user of the client system. Additionally, oralternatively, processing related to tracking frequently requestedresources may include excluding a second resource from the count ofrequested resources if a communication identifying the second resourceas a resource explicitly requested by the user of the client system wasnot received from the client system.

DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram showing a network that includes server computers andclient computers.

FIG. 2A is an illustration showing a web browser and displayed web page.

FIG. 2B is a portion of the source code for the web page shown in FIG.2A.

FIGS. 3A-3C show various HTTP requests related to the web page shown inFIG. 2A.

FIG. 4A is a flowchart showing a process that occurs when a web browserrequests a web page.

FIG. 4B is a flowchart showing a process that may be used to insure thatthe URL, included in the page view field is the URL of a web page orother document that triggered the call for a delivered resource.

FIG. 4C is a flowchart of an alternative process that may be used toinsure that the URL included in the page view field is the URL of theweb page or other document that triggered the call for a deliveredresource.

FIG. 4D shows exemplary HTML code for a frame webpage.

FIG. 4E is diagram illustrating the resulting frames for the source codeshown in FIG. 4D.

FIG. 5 is a flowchart that shows an implementation of proxy serverprocessing.

FIG. 6A is a flowchart that shows an implementation of proxy serverprocessing.

FIG. 6B is a diagram that shows a relationship between multiple URLs forweb pages.

FIG. 7 is a flowchart that shows an implementation of proxy serverprocessing.

FIG. 8 illustrates a flow chart of a process that may alternatively beused for applications in which it is desirable to track the URLsintentionally selected by a user.

FIG. 9 illustrates a flow chart of another process that mayalternatively be used for applications in which it is desirable to trackthe URLs intentionally selected by a user.

DETAILED DESCRIPTION

Generally, a page view field is included in a request for a resource(e.g., an HTTP request for an URL). The page view field includes aheader and data that indicates the URL identifier for the web page orother document used to trigger the request for the resource (eitherdirectly or indirectly). A browser may add the page view field to arequest whenever the requested resource is obtained from within a webpage or other document displayed by the browser, e.g., embedded objects.

Certain processes may be used to help insure that the URL included inthe page view field is the URL of the web page or other document thatinitially triggered a request (i.e., the web page or other document fromwhich the requested URL was obtained, either directly or indirectly). Insuch processes, the URL requested for new navigations by the user (e.g.,when the user types the URL into a navigation bar of a browser or clickson a hyperlink) is stored as a Main Page URL as long as the newnavigation is not a frame navigation. This stored Main Page URL isappended to or otherwise included in subsequent requests that are notnew navigations or frame navigations. Once a new navigation occurs thatis not a frame navigation, the new URL requested as a result of thenavigation is then stored as the Main Page URL and is appended to orotherwise included in subsequent requests that are not new navigationsor are frame navigations.

In this manner, items embedded within a web page and called during thereading of that web page are identifiable as such, and madedistinguishable from other items that are otherwise called. Moreover,this approach enables distinction among resources that are triggered byusers and web page calls, even if both result in population of the HTTPReferer Header. For example, with reference to the example provided inthe background section, the page view field would help distinguish acall to www.aol.com that resulted from user navigation to that site froma call to www.aol.com that results from rendering of another page thatmerely embeds a call to www.aol.com. Significantly, the HTTP RefererHeader may not be useful for similar purpose since the content renderedat www.aol.com may result from a redirect in either case. Moreover, thisability to make this distinction is useful in understanding and/orcontrolling the intentional browsing activities of one or more users.

The page view field may be used by a proxy or other server to performprocessing related to a number of applications. The processing mayrelate to access controls, e.g., parentally controlled accounts. Forexample, the applications may include the reporting of URLs visited byparentally controlled accounts; or allowing parentally controlledaccounts to access lower level web pages of web pages the account hasbeen given permission to access. Also, the processing may relate toaccurately tracking frequently requested resources such as web pages.

Alternatively, or additionally, such distinctions between resourcesexplicitly or directly selected by a user and resources indirectlyselected may be enabled by identifying an explicitly selected web pageor other resource as such in the request for the web page or otherresource, which may allow the web page or resource to be differentiatedfrom web pages or other resources that are requested as a consequence oftheir indirect selection (e.g., by being requested as a result of aresource explicitly requested by a user). Moreover, a log of web pagesor other resources explicitly selected by a user may be maintained atthe client and later reference by a local processor or communicated to ahost process seeking to differentiate directly and indirectly selectedweb pages or other resources. Such communications may be made in acommunication separate from requests for resources. These techniquesalso may allow a proxy or other server to perform processing related toparentally controlled accounts or related to accurately trackingfrequently requested resources such as web pages.

FIG. 1 shows a network 100 that includes server computers 131-133 andclient computers 111-113. Server computers 131-133 may execute hypertexttransfer protocol (HTTP) server software to respond to data requestsfrom HTTP-based web browsers executing at client computers 111-113.Client computers 111-113 can send HTTP data requests to servers 131-133over data paths that include access connections 114-116, a serviceprovider's point of presence (POP) 110, network 120, proxy server 117,and a public network 130. The HTTP data requests can target varioustypes of resources such as documents (e.g., web pages) or objects (e.g.,executable files, audio files, or video files). The service provider'sPOP 110 includes data communications equipment that enables andregulates communication between client computers 111-113 and the serviceprovider's network 120. For example, POP 110 may include dial-up modembanks, cable modem banks, wireless communications equipment, or otherdata transmission equipment.

Access to a service provider's POP 110 may be restricted to certainusers of the client computers 111-113. To enforce access restrictions,POP 110 may implement security and authentication mechanisms such aslogin verification. A login verification mechanism may require a user toinput a valid user name and password to obtain access to the serviceprovider's network 120. If the user name and password are invalid, theuser may be disconnected. Security and authentication mechanisms alsomay be implemented at a separate login server (not shown) connected tothe service provider's network 120 and/or POP 110.

After a valid connection has been established between a client computerand POP 110, data may be exchanged between software applications runningon the client computer, and applications running on other computers onthe service provider's network 120. The service provider's network 120may be interconnected with another network 130 by a proxy server 117that can exchange data between the service provider's network 120 andcomputers on another network 130.

Proxy server 117 can function as a surrogate for another computer. Forexample, proxy server 117 may receive HTTP data requests directed toHTTP server software at one of the server computers 131-133 from abrowser application at client computer 111. When the proxy server 117receives a data request, it may attempt to fulfill the data requestusing data stored at a local database or hard disk (not shown). If theproxy server 117 has the needed data, the data can be returned to theclient computer 111 without requiring further interaction with servers131-133. If the proxy server 117 is unable to fulfill the request, itmay forward the request to the appropriate one of the servers 131-133,receive a response from that server, and send the response to the clientcomputer 111. Proxy server 117 may perform some processing on theresponse before sending it to client computer 111.

The proxy server 117 also may store response data on a hard disk driveor local database for future use. Networks 120 and 130 also may beinterconnected by a gateway, bridge, router, or other interconnectiondevice instead of, or in addition to, proxy server 117.

While the foregoing has described the various components of network 100as using the HTTP protocol, other standard or proprietary communicationprotocols may alternatively be used. In another implementation, forexample, proprietary browsing software may execute on client computers111-113 and may communicate with other computers (including proxycomputer 117) on service provider network 120 using proprietaryprotocols, or a mix of standard and proprietary protocols. Proxy server117 interfaces service provider network with public network 130 bytranslating requests and responses from the proprietary protocol intothe standard protocols. Additionally, if the browser executing on clientcomputer 111 only renders web pages written in a proprietary languageand a web page written in a standard language is retrieved from a server131-133, proxy server 117 may convert the standard language web pageinto a proprietary language web page.

Thus, in this exemplary implementation, client computers 111-113 executethe proprietary browsing software and communicate with proxy server 117using the proprietary communications protocol. Proxy server 117 thencommunicates with servers 131-133 using the standard protocol to receivethe requested web page. Proxy server 117 translates the web page asnecessary and forwards the web page to the appropriate one of the clientcomputers 111-113 using the proprietary communications protocol.

In other implementations, public network 130 may use other standard orproprietary communication protocols and web pages may be written inother standard or proprietary languages. In general, the standard orproprietary protocol or language used on public network 130 may be thesame or different than the standard or proprietary protocol or languageused on service provider network 120.

FIG. 2A shows a web browser 200 display of a web page 210. The term webpage as used herein refers generally to documents displayed by abrowser, whether those documents are written in a standard orproprietary language.

Web page 210 is the default web page for the domain www.cnn.com. Afterproxy server 117 has sent web page 210 to, e.g., client computer 111(either from the proxy server's local hard drive or database or from aserver 131-133), web browser 200 renders the web page 210 so that a userof client computer 111 can view it. Web page 210 is composed of text,hyperlinks and a number of objects such as embedded graphics. Forexample, web page 210 includes graphics 212-224, hyperlinks 226, andtext 228. While not shown, the objects can also include, for example,audio files, video files, executable files, or other resources.

Web pages are typically text files written in a standard or proprietarylanguage that is understood by browser 200. Some standard languagesinclude HTML and the extended mark-up language (XML). The text file(otherwise referred to as the web page source code) constitutesinstructions to the web browser 200 as to what the browser shoulddisplay when it renders the web page. The web browser 200 processes thesource code (e.g., a HTML text document) and renders the web page 210 tothe user accordingly.

Because web pages are normally text files, they typically contain textand hyperlinks. By contrast, graphics typically need to be retrievedwhen browser 200 renders a web page. The web page source code includesinstructions that direct browser 200 to the location of the graphics tobe loaded (typically located on the server that provided the web pagesource code). Browser 200 uses the instructions to retrieve the graphicswhen browser 200 renders the web page.

FIG. 2B shows a portion of source code 230 that corresponds to web page210. The source code for web page 210 is written in HTML. HTML consistsof text “tags” that provide browser 200 with certain information andinstruct the browser 200 how to display that information. A tag is textsurrounded by the brackets “o>.” In the example provided by source code230, tags 232-236 primarily instruct browser 200 as to where browser 200is to display graphic 212. Tags 238 a and 238 b instruct browser 200 todisplay graphic 212 and to hyperlink the graphic 212 to a particular webpage (in this case, graphic 212 is hyperlinked to web page 210). Theanchor tag 238 a, <a>, instructs the browser to create the hyperlink.The image tag 238 b, <img>, instructs the browser to insert graphic 212.Image tag 238 b includes the location of graphic 212 as an argument. Thelocation of the graphic 212 is expressed as a Uniform Resource Locator(URL) 240.

URL 240 includes several sections that identify the location of graphic212. A first section 242 indicates that graphic 212 is located at aserver whose domain name is “i.a.cnn.net.” First section 242 alsoindicates that graphic 212 is available at the server via the HTTPprotocol. A second section 244 indicates the directory location ofgraphic 212 on the server. A third section 246 indicates the filename ofgraphic 212, namely, “logo.gif.”

Using the URL 240 and the domain name system (DNS), web browser 200 candetermine the address of the server computer (e.g., server computer 131)that stores graphic 212, along with the location of graphic 212 on theserver computer 131. With this information, web browser 200 can retrievegraphic 212 from server computer 131 and display graphic 212appropriately.

More specifically, when browser 200 is rendering web page 210, browser200 processes image tag 238 b by first contacting a DNS server (notshown) to obtain the address of the server whose domain name is“i.a.cnn.net” (e.g., server computer 131). Next, browser 200 causesclient computer 111 to send a request for graphic 212 to server computer131, which is first received by proxy server 117. Proxy server 117 mayperform some processing as a result of receiving the request (furtherdescribed below). Proxy server 117 then forwards the request for graphic212 to the server computer 131, receives graphic 212 from servercomputer 131, and forwards graphic 212 back to client computer 111 sothat browser 200 can display graphic 212.

Other implementations of the steps used to obtain web pages or otherinformation from servers 131-133 are possible, and may depend on theparticular protocols used by client computers 111-113, proxy server 117,and severs 131-133.

FIG. 3A shows an HTTP request 300. In an HTTP implementation, an HTTPrequest 300 is sent by client computer 111 to proxy server 117. HTTPrequest 300 includes a structured sequence of fields 302-306. Each field302-306 includes an HTTP header and data associated with the header. Forexample, field 302 includes the header “GET,” which indicates that theHTTP request is a “GET” request to obtain a default web page located atthe top-level of the directory structure (indicated by data “/”) at theserver. In field 304, header “Host:” indicates the server computer fromwhich the default web page is to be obtained. In request 300, the serveris the server whose domain is “www.cnn.com.” Field 306 includes the HTTPheader “User-Agent:.” The data of field 306 is “Mozilla/4.0 (compatible;MSIE 6.0; Windows NT 5.0; Q312461),” which designates the type ofbrowser generating the request 300.

In addition, HTTP requests are sent to proxy server 117 to retrieve theobjects, such as graphics 212-224, included in web page 210 when thebrowser renders the web page. For example, FIGS. 3B and 3C show the HTTPrequests 310 and 320 for graphics 212 and 216, respectively. Request 310has fields 312-318 and request 320 has fields 322-326. Fields 312-316and 322-326 are similar to fields 302-306 of request 300. For example,field 312 indicates that the request is a GET request to obtain graphic212 and includes the filename and location of graphic 212 on the server,namely, “/cnn/.element/img/1.1/logo/logo.gif.” Field 314 indicates thegraphic 212 is located on the server whose domain name is “i.a.cnn.net”and field 316 indicates the type of browser making the request.

Each of requests 310 and 320 also include an additional field 318 and328 (the page view field), respectively. This field includes a headerand data that indicates the URL for the web page or other document fromwhich the requested URL was obtained. For example, in request 310, field318 includes the header “X-PageView:” that has as its data“www.cnn.com/.” The data is “www.cnn.com/” because requested URL 240(which points to graphic 212) was obtained from the default web page 210on www.cnn.com and the so default web page 210 for www.cnn.com has theURL “www.cnn.com/.” Browser 200 obtained requested URL 240 for graphic212 from this default web page 210 when the browser 200 was renderingweb page 210. Similarly, field 328 includes the header “X-PageView:”that has as its data “www.cnn.com/.”

As can be en by comparing requests 310 and 320, some objects on webpages are not from the same servers, or even from the same domain. Forexample, referring to fields 314 and 326, graphic 212 is located on theserver with the domain “i.a.cnn.net,” while the graphic 216 is locatedon the server with the domain “spd.atdmt.com.”

As described above, HTTP requests 300, 310 and, 320 may be sent from aclient computer 111 to proxy 117 and forwarded through the proxy 117 fordelivery to a server computer 131. As further described below, the pageview field in requests 310 and 320 allow proxy server 117 to performprocessing based on a page view (as described below) of a web page 210as an alternative, or in addition to, an object view of the web page, asopposed to simply being able to perform processing on an object view.

Referring to FIG. 4A, when a page view field is used, web browser 200generates a request (405) and adds the page view field to the request(410), if appropriate. Browser 200 typically adds the page view fieldwhenever the URL for the information being requested is obtained fromwithin a web page or other document displayed in browser 200. Thus, forexample, the page view field is added when the browser is rendering aweb page and needs to retrieve an object for display in the page, or,for example, when a user selects a hyperlink in a web page.

The URL for the information being requested may be obtained directlyfrom within the web page or other document being displayed, orindirectly, for example, as a result of a HTTP redirect. For instance, aURL in a web page may point to a redirecting resource that returns theURL to a graphic that is to be displayed in the web page. Browser 200may track that the URL to the graphic is obtained (although indirectly)from the URL it the web page. Browser 200 then includes the URL for theweb page in the page view field that is part of the request for thegraphic, instead of the URL to the redirecting resource from which theURL for the graphic was directly obtained.

The request then is transmitted to proxy server 117 (415), whichperforms processing using the page view field (420). The page view fieldallows proxy server 117 to have a page view of the requests received byproxy server 117, thus providing proxy server 117 with information aboutthe existence (or lack thereof) and location/identification of a webpage containing the requested URL. This allows proxy server 117 toperform processing based on the fact that the requested URL (or objectpointed to by the requested URL) is part of a web page and which webpage the requested URL (or object pointed to by the requested URL) is apart of. Proxy server 117 also may perform processing based on theobject view of the requests. An object view refers to the informationthat the proxy server has about the URL itself or information about theobject itself pointed to by the URL, without regard to the relationshipbetween the URL or object and other web pages.

The proxy server processing may relate to a number of differentapplications. As described below, the processing may relate toparentally controlled accounts. For example, the applications mayinclude the reporting of URLs visited by parentally controlled accounts;or allowing parentally controlled accounts to access lower level webpages of web pages the account has been given permission to access. Alsoas described below, the processing may relate to accurately trackingfrequently requested resources such as web pages.

Proxy server 117 optionally removes the page view field from the request(425) if it is included, and forwards the request to the appropriateserver computer 131-133 (430). Because the page view field is not astandard field, servers do not use this field, and therefore, removingit does not effect server processing of the request. The appropriateserver computer 131-133 processes the request, as normal, and returned areply to proxy server 117, which forwards the reply to the appropriateclient computer 111-113.

FIG. 4B is a flowchart of a process 440 that may be used for actions405, 410, and 415 to insure that the URL included in the page view fieldis the URL of the web page or other document that initially triggered arequest (i.e., it is the URL of the web page or document from which theURL for the information was obtained, either directly or indirectly).Browser 200 generates a request (442). A determination is made as towhether the request was generated as a result of a new user navigation(e.g., when the user types a URL into a navigation bar of browser 200 orclicks on a hyperlink) (444). If the request is a result of a newnavigation (444), then, optionally, a determination is made as towhether the new navigation is a frame navigation (446).

A frame navigation is one in which a new web page is loaded into a frameof a currently displayed frame web page. When using HTML, an author cangenerate frame web pages that have two or more frames, or sub-windows,within the displayed web page. Each of the frames may be loaded with adifferent web page for display to the user. For example, within a webpage, one frame might display a static banner, a second frame maydisplay a navigation menu, and a third frame may display the maindocument that can be scrolled through or replaced by clicking on ahyperlink in the navigation menu contained in the second frame.

The source code of a frame web page contains instructions that directthe browser to split the display into two or more frames duringrendering and to retrieve items (e.g., other web pages) for display ineach frame. When processing the source code of a frame web page duringrendering, the browser splits the display into multiple frames. For eachframe, the browser then obtains the item that is to be displayed in theframe. If the item is a web page, the browser obtains the web page andbegins rendering the web page in the appropriate frame. The user maythen, for example, click on a hyperlink in the web page displayed in theframe or in another frame to cause a new web page or other resource tobe displayed in either of the frames. This navigation is considered tobe a frame navigation, since the new web page is loaded into one of theframes of the displayed web page.

As an example, FIG. 4D shows exemplary web page source code 476 thatresults in a frame web page 478, such as that shown in FIG. 4E. Thesource code instructs the browser to split the display into threeframes, namely, Frame 1, Frame 2, and Frame 3. The source code alsoinstructs the browser to request and render a first web page(“contents_of_frame1.html”) into Frame 1, a second webpage(“contents_of_frame2.html”) into Frame 2, and a third webpage(“contents_of_frame3.html”) into Frame 3. When the third webpage isdisplayed in Frame 3, the user may, for example, click on a hyperlink inthe third webpage and a new webpage (corresponding to the hyperlink) isloaded into Frame 3. The user also may be able to click on a hyperlinkin the second webpage loaded in Frame 2 and the new webpage(corresponding to the hyperlink) is loaded into Frame 3. These twoinstances are frame navigations. The user additionally may be able totype the URL of a new web page into a navigation bar of the browser, orclick a hyperlink in one of the webpages, and the new web page is openedin the full display of the browser (i.e., it is not loaded into one ofthe frames, rather it is opened as its own webpage in the browser). Sucha navigation is not a frame navigation.

As described above, in process 440, if the request is a result of a newnavigation (444), then a determination is made as to whether the newnavigation is a frame navigation (446). If the new navigation is not aframe navigation, then the URL requested in the request generated by thebrowser is stored as a Main Page URL (452) and the request is sent tothe server (452). Subsequent requests that are not new navigations, orthat are new navigations but are not frame navigations, have this storedMain Page URL appended to or otherwise included with them in the pageview header.

This is illustrated by process 440 with respect to action 450. Thus, asshown, when the browser request is not a new navigation (444), thepreviously stored Main Page URL is appended to or otherwise included inthe request in the page view header (450). Similarly, even if therequest is a new navigation (444), if it also is a frame navigation(446), the previously stored Main Page URL is appended to or otherwiseincluded in the request in the page view header (450) and the request issent to the server (452).

In other implementations, the URL requested for a frame navigation mayalternatively be stored as the Main Page URL. That is, a new navigationresults in the requested URL being stored as the Main Page URL orotherwise used as a basis for page view header, regardless of whether ornot the navigation is a frame navigation.

As an example of process 440 applied to web page 210, a user enters theURL for webpage 210 (i.e., “www.cnn.com”) into a navigation bar ofbrowser 200. Browser 200 generates a request as a result (442) (whichwill be sent from client computer 111 to, for example, server computer132). This request is the result of a new navigation (444), that is nota frame navigation (446). Consequently, the URL “www.cnn.com” is storedas the Main Page URL (448). However, the request is sent to the servercomputer 132 without having www.cnn.com included within a page viewheader (452). The request is first received by proxy server 117. Proxyserver 117 then forwards the request for webpage 210 to the servercomputer 132, receives the source code for webpage 210 from servercomputer 132, and forwards the source code for webpage 210 back toclient computer 111 so that browser 200 can render webpage 210.

Browser 200 begins processing the source code and rendering webpage 210.As part of rendering webpage 210, browser generates requests for objectsin webpage 210, such as graphics 212 and 216. For example, browser 210begins to render graphic 212 before graphic 216. As a result, browser210 generates a request for graphic 212 (442) (which will be sent fromclient computer 111 to, for example, server computer 131). This requestis not the result of a new navigation (444) because it is beingperformed as part of the rendering. Consequently, the stored Main PageURL (“www.cnn.com”) is appended to or otherwise included in the requestin the page view header of the request or otherwise included within arequest (450) that is sent to server computer 131 (452). The request isfirst received by proxy server 117, which may perform some processing asa result of receiving the request (further described below). Proxyserver 117 then forwards the request for graphic 212 to the servercomputer 131, receives graphic 212 from server computer 131, andforwards graphic 212 back to client computer 111 so that browser 200 candisplay graphic 212.

Next, browser 210 begins rendering graphic 216. As with graphic 212,browser 210 generates a request for graphic 216 (442) (which will besent to, for example, server computer 133). This request is not theresult of a new navigation (444) because it is being performed as partof the rendering. Consequently, the stored Main Page URL (“www.cnn.com”)is appended or otherwise included as the page view header of the requestor otherwise included within a request (450) that is sent to servercomputer 133 (452). Again, the request is first received by proxy server117, which may perform some processing as a result of receiving therequest (further described below). Proxy server 117 then forwards therequest for graphic 216 to the server computer 133, receives graphic 216from server computer 133, and forwards graphic 216 back to clientcomputer 111 so that browser 200 can display graphic 216.

Once web page 210 is rendered, if the user, for example, clicks on ahyperlink (e.g., one of the hyperlinks 226), then browser 200 generatesa request (442). This request, however, is the result of a newnavigation (444) and is not a frame navigation (446). Consequently, theURL pointed to by the hyperlink the user clicked on is stored as theMain Page URL (448) and the request is sent to the appropriate servercomputer without having a page view header appended thereto or otherwiseincluded therewith (452).

FIG. 4C is a flowchart of an alternative process 460 that may be usedfor actions 405, 410, and 415 to insure that the URL included in thepage view field is the URL of the web page or other document that causedthe information to be requested. Process 460 is particularly suited tobe implemented by a plug-in to browser 200 to provide browser 200 withthe ability to append page view headers to requests as appropriate. Inother words, in one implementation, browser 200 does not implement thefunctionality to append the page view header to requests. Rather, aplug-in to browser 200 implements the functionality to appropriatelyappend or otherwise include the page view header. Process 460 isparticularly suited to be performed by such a plug-in.

Accordingly, when there is a new navigation by a user, browser 200provides an indication to the plug-in that a new navigation has occurred(462). The plug-in then queries browser 200 to determine whether the newnavigation is a frame navigation (464). If the new navigation is not aframe navigation (466), then the URL requested in the new navigation isstored by the plug-in as the Main Page URL (468). Then, when the browsergenerates a request as part of rendering, the browser provides anindication to the plug-in (470), which then instructs browser 200 toappend the stored Main Page URL in the request (472). The plug-in thenwaits for new indications of a new navigation, or a new request that isnot a new navigation, and responds accordingly.

When the new navigation is a frame navigation (466), then the plug-ininstructs browser 200 to append a previously stored Main Page URL to therequest. Then, when the browser generates a request as part ofrendering, the browser provides an indication to the plug-in (470),which then instructs browser 200 to append the stored Main Page URL tothe request (472). In this case, the stored Main Page URL is still thepreviously stored Main Page URL because a new navigation that is not aframe navigation has not occurred. The plug-in then waits for newindications of a new navigation, or a new request that is not a newnavigation, and responds accordingly.

As an example of process 460 applied to web page 210, a user enters theURL for webpage 210 (i.e., “www.cnn.com”) into a navigation bar ofbrowser 200. Browser 200 generates a request as a result (which will besent from client computer 111 to, for example, server computer 132) andprovides an indication to the plug-in of the new navigation, along withthe URL of the new navigation (462). The plug-in then queries browser200 to determine whether the new navigation is a frame navigation (464).This request is not a frame navigation (466) and, consequently, the URL“www.cnn.com” is stored by the plug-in as the Main Page URL (468).However, the request is sent to the server computer 132 without havingwww.cnn.com included within a page view header. The request is firstreceived by proxy server 117, which then forwards the request forwebpage 210 to the server computer 132, receives the source code forwebpage 210 from server computer 132, and forwards the source code forwebpage 210 back to client computer 111 so that browser 200 can renderwebpage 210.

Browser 200 begins processing the source code and rendering webpage 210.As part of rendering webpage 210, browser generates requests for objectsin webpage 210, such as graphics 212 and 216. For example, browser 210begins to render graphic 212 before graphic 216. As a result, browser210 generates a request for graphic 212 (which will be sent from clientcomputer 111 to, for example, server computer 132). Browser 200 providesan indication of the request to the plug-in (470), which then instructsbrowser 200 to append the stored Main Page URL (i.e., “www.cnn.com”) tothe request (472). Browser 200 appends the Main Page URL to the requestas the page view header of the request or otherwise includes it within arequest that is seat to server computer 131. The request is firstreceived by proxy server 117, which may perform some processing as aresult of receiving the request (further described below). Proxy server117 then forwards the request for graphic 212 to the server computer131, receives graphic 212 from server computer 131, and forwards graphic212 back to client computer 111 so that browser 200 can display graphic212.

Next, browser 210 begins rendering graphic 216. As with graphic 212,browser 210 generates a request for graphic 216 (which will be sent to,for example, server computer 133). Browser 200 provides an indication ofthe request to the plug-in (470), which then instructs browser 200 toappend to or otherwise include with the request the stored Main Page URL(i.e., “www.cnn.com”) (472). Browser 200 appends includes the Main PageURL to the request in the page view header and sends the request toserver computer 133. The request is first received by proxy server 117.Proxy server 117 may perform some processing as a result of receivingthe request (further described below). Proxy server 117 then forwardsthe request for graphic 216 to the server computer 133, receives graphic216 from server computer 133, and forwards graphic 212 back to clientcomputer 111 so that browser 200 can display graphic 216.

Once web page 210 is rendered, if the user, for example, clicks on ahyperlink (e.g., one of the hyperlinks 226), then browser 200 generatesa request and provides an indication of a new navigation to the plug-in(462). The plug-in queries browser 200 to determine if this newnavigation is a frame navigation (464), which it is not (466).Therefore, plug-in stores the URL corresponding to the hyperlinkselected as the Main Page URL (468) and process 460 continues.

In processes 440 and 460, when a request is the result of a newnavigation that is not a frame navigation (and, hence, the requested URLis stored as the Main Page URL), the above description indicates that apage view header will not be appended or otherwise included. However, asan alternative, it is possible, and in fact contemplated, that therequested URL may be appended to or otherwise be included with therequest in the page view header.

Processes 440 and 460 insure that the URL of the web page whoserendering caused an object to be retrieved is included in the page viewheader of the request for the object. Thus, unlike implementations ofthe HTTP referrer, processes 440 and 460 can provide a page view (i.e.,an understanding of what objects are part of a web page) to, forexample, a proxy server. Having a page view is useful in a number ofapplications, such as those described below. Because of the HTTPreferrer's inconsistent information, it was found to be unsuitable forat least these applications and it does not appear to have been designedfor applications that use a page view of the objects being retrieved bya browser.

As described, proxy server processing may relate to the reporting ofURLs visited by parentally controlled accounts. Some Internet ServiceProviders (ISPs) provide parents with the ability to create accountsthat are parentally controlled, i.e., have certain permissions set bythe parents. As part of the parental control service, an ISP may wish toprovide an indication of the URLs visited by the parentally controlledaccount and report the URLs to a parental or supervisory account, forexample, by sending an e-mail to the parental account with a list of theURLs visited; by presenting a dialog box to a user of the parentalaccount with a list of the URLs; or by presenting a web page to the userof the parental account with a list of the URLs. The user of theparental or supervisory account, or of the parentally controlledaccount, may be a child, a parent or supervisor, or both.

The proxy server 117 may use the page view field to more accurately showwhich URLs were retrieved intentionally by the user of the parentallycontrolled account. With the page view field, proxy server 117 candetermine that the requested URL was obtained from the web page when theweb page was displayed to the user. To help prevent confusion on thepart of the parents, proxy server 117 may exclude the URLs obtained fromthe web page. That is, to avoid misleading a parent with an indication,for example, that the user of the parentally controlled accountintentionally visited a default web page and all of the objects (e.g.,graphics) displayed, embedded, or otherwise called because of thedefault web page, the web page view may be used to distinguish or filterweb pages called as a result of an accessed web page being rendered.

For example, if the user of the parentally controlled account retrievesthe default web page at www.cnn.com/, proxy server 117 stores this URLwhen HTTP request 300 is received from, e.g., client computer 111. Asweb browser 200 is rendering web page 210, proxy server 117 alsoreceives HTTP requests 310 and 320. Without the page view field, proxyserver 117 may be unable to determine that the URLs in requests 310 and320 are being requested as a result of web page 210 being rendered, thusproxy server 117 may store each URL and generate a reporting list forthe parents that includes:

-   -   www.cnn.com/    -   i.a.cnn.net/cnn/.element/img/1.1/logo/logo.gif    -   spd.atdmt.com/b/AANYCVCSTVST/1CNN_1028_728x90.jpg

While it is possible that the parents may be able to guess that thesecond URL resulted from the user's visit to the web page atwww.cnn.com/ (because the URLs include similar domains, namely, cnn.comand cnn.net), they will be burdened to do so. Furthermore, it is likelydifficult for the parents to determine that the third URL was requestedas a result of the user visiting www.cnn.com/. Thus, the parents mayfalsely believe that the user intentionally visited the third URL, whenin fact the URL is listed because the user visited www.cnn.com/. Thismay lead parents to believe their children are visiting many more sitesthan the children actually are, or visiting sites that the parentspotentially may not approve of.

By using the page view header, however, proxy server 117 can determinethat the second and third URLs were obtained from within web page 210(at www.cnn.com/). As a result, proxy server may exclude these URLs fromthe list reported to the parents.

To differentiate between URLs retrieved when rendering a web page andURLs retrieved when a user selects a hyperlink in the web page, the pageview header may have an additional data item that indicates whether theURL is one selected by the user or is one that is retrieved whilerendering a web page. In such an implementation, proxy server 117 mayexclude those URLs that are contained in a web page and retrieved as aresult of the web page rendering, but include in the reporting listthose URLs that are contained in a web page as hyperlinks and selectedby the user. This may be useful because parents, for example, may beinterested in the web pages the user of the parentally controlledaccount is intentionally visiting (e.g., by selecting a hyperlink or bytyping a URL into the web browser 200). But at the same time, theparents may not want to know the URLs of the objects displayed in theintentionally visited web pages, or may be confused by those additionalURLs being included, as described above. Thus, for example, it may bedesirable to exclude the URL for graphic 212 from the list, but includethe URL for a hyperlink 226 if the user selects the hyperlink.

Referring to FIG. 5, in such an application, the processing based on thepage view header (420) may include determining if a page view field isincluded in the received request (505). If the page view field is notincluded in the request (505), then the requested URL is the onerequested by the user (e.g., by clicking on a hyperlink or typing theURL into an address bar of the browser). Thus, if the page view field isnot included in the request (505), the requested URL in the request isstored for reporting (515), if the URL has not been stored previously(510). If the requested URL has been stored previously (510), then proxyserver 117 forwards the request (430).

If the page view field is included in the received request (505), adetermination is made as to whether the URL contained in the data of thepage view field has been stored for inclusion in the list (525). If not,the URL contained in the data of the page view field is stored (530) andproxy server 117 moves on to removing the page view field (425). If theURL in the data of the page view field has already been stored (525),then proxy server 117 moves on to removing the page view field (425).

Optionally, if the page view field has additional data indicatingwhether the URL was retrieved as a result of being user-selectedselected or as a result of rendering a user-selected web page, thenproxy server 117 may determine whether the request is a result of thepage rendering (520) before determining if the URL in the data of thepage view has previously been stored (525). That is, proxy server maydetermine whether the requested URL is being retrieved as a result ofrendering or as a result of the user selecting a hyperlink for therequested URL (520). If the requested URL is being retrieved based onthe user selecting a hyperlink (520), then proxy server 117 continuesthe previous processing by determining whether the URL contained in thedata of the page view field has been stored for inclusion in the list(525). If, on the other hand, the requested URL is being retrieved as aresult of rendering (520), then proxy server 117 moves on to removingthe page view field (425).

Another implementation (an example of which is described with respect toFIGS. 6A and 6B) involves allowing parentally controlled accounts toaccess lower level web pages of web pages the account has been givenpermission to access. As part of a parental control service, an ISP mayblock particular web pages from being visited by parentally controlledaccounts. The ISP may give the user of the parentally controlled account(typically a child or young teen) the ability to request from theparents that certain web pages be unlocked, i.e., that the parentallycontrolled account be granted permission to view the otherwise blockedweb pages.

Proxy server 117 may be used to provide the blocking service.Specifically, proxy server 117 may receive an HTTP request and comparethe requested URL to a list of blocked URLs for the parentallycontrolled account. When parents grant permission to visit a certain webpage, the web page is removed from the list of blocked sites.Alternatively, a list of excluded sites may be kept for parentallycontrolled accounts generally, and each parentally controlled accountmay have a specific list of web pages for which permission has beengranted.

At times, a parent may want to grant permission for the account to beable to access a top-level web page and all of the lower level web pageshyperlinked to in the top-level web page. For example, a parent may wishto give the account access to the web page at “www.cnn.com/” and the webpages connected thereto by hyperlinks 226. Yet, it would be burdensometo the parent to indicate every web page that they wish to givepermission to visit. At the same time, without the URL for the web pagebeing explicitly designated by the parent, it may be difficult for proxyserver 117 to determine if the lower level web pages should be permittedbased simply on reviewing the URL for the lower level web page includedin the request. That is, if the lower level web pages are typicallyblocked, it may not be possible for proxy server 117 to determine thatthey should now be allowed simply by reviewing the URL in the requestfor the lower level web page. This is particularly the case if the lowerlevel web page does not share the same domain as the top-level web page.

Using the page view field, however, proxy server 117 can determine thatthe URL for the lower level web page was obtained from a permitted pageand, therefore, should be permitted also. Thus, by comparing the URL inthe data of the page view request to the list of allowed sites or to thelist of excluded sites (depending on the implementation), proxy server117 can determine whether the object pointed to by the requested URLshould be retrieved and forwarded to the client computer.

Referring to FIG. 6A, in one example of such an application, theprocessing based on the page view header (420) may include determiningif a page view field is included in the received request (605). If thepage view field is not included in the request, proxy server 117determines whether the parentally controlled account is permitted toretrieve the item at the requested URL (610). If so, proxy server 117obtains the item at the requested URL and forwards the item to theclient computer (615). Otherwise, proxy server 117 does not obtain theitem and may optionally inform client computer that the parentallycontrolled account is not permitted to access the item (620).

If the page view field is included, a determination is made as towhether the parentally controlled account is permitted to access theitem at the URL contained in the data of the page view field (i.e., thereferring page) (625). If so, then the item at the requested URL isobtained and forwarded to the client computer (615). If not, proxyserver 117 optionally may make a separate determination as to whetherthe parentally controlled account can access the requested URL (630).This may be useful in situations in which a web page the account doesnot have permission to access contains a hyperlink to a web page theaccount does have permission to access. If proxy server 117 determinesthat the parentally controlled account can retrieve the URL, proxyserver 117 obtains the item at the requested URL and forwards the itemto the client computer (615). Otherwise, proxy server 117 does notobtain the item and may optionally inform client computer that theparentally controlled account is not permitted to access the item (620).

Optionally, proxy server 117 may store the requested URLs and theirrelationship among each other and determine from this relationshipwhether access to the requested URL should be granted. The web pageslinked to by a permitted web page may themselves link to other webpages, all of which constitute the same site. Parents may want to givepermission to the parentally controlled account to access the entiresite. For example, a parentally controlled account may be givenpermission to visit the cnn.com website. The cnn.com website includesweb page 210 and the web pages connected to web page 210 by hyperlinks226. The web pages connected to web page 210 also may have hyperlinksconnecting to other web pages on the cnn.com web site. By storing whichURLs are contained in which web pages, at least during the currentsession, proxy server 117 may appropriately provide access to the lowerlevel web pages.

Referring to FIG. 6B, for example, the web page 210 at URL 640(www.cnn.com/) contains a hyperlink 226 a for the webpage (not shown) atURL 645 (http://www.cnn.com/2003/LAW/11/25/jackson.case/index.html),which contains a hyperlink (not shown) for the web page (not shown) atURL 650(http://www.cnn.com/2003/LAW/11/20/otsc.toobin.lalama/index.html). Ifthe parents give permission to access the web page at URL 640 and theparentally controlled account accesses the web page at URL 645 viahyperlink 226 a, then the data in the page view field will contain URL640 and, consequently, proxy server 117 will retrieve the web page atURL 645. If the account, however, attempts to access the web page at URL650 via a hyperlink on the web page at URL 645, then the data in thepage view field will contain URL 645, which has not been specificallypermitted. Thus, proxy server 117 may not permit access to the web pageat URL 650. However, if proxy server 117 stored the information that URL645 was obtained from the web page at URL 640, and that the web page atURL 640 is permitted, proxy server 117 may permit access to the web pageat URL 650 because the parents gave permission for access to the websiteat cnn.com and the web page at URL 650 is connected to the top-levelpage of the website cnn.com.

Other applications also may benefit from tracking the relationship amongrelated URLs. For example, the application to reporting visited webpages may track the relationships between related URLs so that only thetop-level web page (or web page manually entered) is reported. That is,the web pages whose URLs are obtained from within a web page (whetherretrieved as a result of rendering or as a result of the user selectinga hyperlink) may be excluded from the report to the parental account.Alternatively, a pre-selected default or user-designated number of lowerlevel web pages viewed may be reported with the top-level web page.

Alternatively, proxy server 117 may allow a parentally controlledaccount to access web pages at URLs located on a web page alreadyretrieved, i.e., it is assumed that if the parentally controlled accounthas permission for a web page, then it has permission for lower levelweb pages or other objects hyperlinked to the permitted web page. Forexample, because the web page for URL 645 was already retrieved (becauseit was determined to be permitted), when the parentally controlledaccount requests the web page at URL 650 by a hyperlink in the retrievedweb page at URL 645, proxy server 117 observes that URL 645 is containedin the data of the page view field and, consequently, obtains the webpage at URL 650 because proxy server 117 has already obtained the webpage at URL 645.

Another application (an example of which is described with respect toFIG. 7) entails accurately tracking popular web pages. Some ISPs provideits members with a list of the web pages that are visited the most bythe members of the ISP over a particular period of time. The ISP trackswhich web pages are being requested by the members of the ISP, ranks therequested web pages based on the number of requests received in acertain time, and displays a certain number (e.g., 5) of the highestranking web sites to the users. The ISP tracks the web pages based onthe URLs requested by its users. To do so, for example, when clientcomputer 111 sends request 300 for the web page at www.cnn.com/, proxyserver 117 stores the URL requested in request 300 (i.e., www.cnn.com/),along with a count of one (assuming client computer 111 is the firstcomputer to request the URL). As other client computers request the URLwww.cnn.com/ during a certain period of time, proxy server 117increments the count accordingly. Proxy server 117 can increment thecount for each time the URL is requested, or for each time a uniqueclient computer requests the URL. Proxy server 117 can track whether aparticular client computer has previously requested the web page by, forexample, noting the IP or other address (or logon information, such asscreen name) of the client computers that have requested the web page.After the certain period of time, proxy server 117 ranks the count ofthe URL www.cnn.com with other URLs requested during that period anddisplays the top (e.g., 5) requested URLs during the time period.

To accurately track which web pages the ISP's members are actuallyrequesting (and hence, which ones are popular, i.e., requested by theusers the most over a certain period of time), it is advantageous forthe ISP to distinguish web pages the members request from, e.g., theobjects requested when a web page is rendered. For example, when clientcomputer 111 sends request 300 for web page 210, client computer 111also will send requests 310 and 320 when web page 210 renders. However,tracking the URLs in requests 310 and 320 does not accurately reflectwhat is popular among the members of the ISP because members of the ISPhave not specifically requested these objects themselves, rather theywere requested as part of the web page at www.cnn.com/. Thus, to reportthese URLs as popular may confuse members into thinking the objects(e.g., graphic 212) are what is popular, instead of the web page atwww.cnn.com/. Further, when only a particular number of popular webpages are displayed, these objects may be ranked higher than web pagesusers intentionally requested, thereby preventing web pages that arepopular from being displayed to the members.

Referring to FIG. 7, in one example of such an application, theprocessing based on the page view header (420) may include determiningif a page view field is included in the received request (705). If not,the URL is stored (if not previously (710)) and a count is set to one(715). If the URL has been previously stored, the count is incrementedby one (720). If proxy server 117 tracks unique requests, then proxyserver 117 may determine whether this is a unique request beforeincrementing the count. Proxy server 117 then moves on to forwarding therequest (430).

If a page view field is included in the request, the URL is not storedor the counter associated with the URL is not incremented Proxy server117 then moves on to removing the page view field (425).

Among other features, FIGS. 1-7 describe appending tracking informationto requests for resources that are requested as a consequence of theirrelationship to a web page or other resource explicitly selected by auser and, thus, which are only indirectly selected by the user.Alternatively, or additionally, the web page or other resourceexplicitly selected by the user may be identified as such in the requestfor the web page or other resource, which may allow the web page orresource to be differentiated from web pages or other resources that arerequested as a consequence of their indirect selection. Moreover, a logof web pages or other resources explicitly selected by a user may bemaintained at the client and later reference by a local processor orcommunicated to a host process seeking to differentiate directly andindirectly selected web pages or other resources. Such communicationsmay be made in a communication separate from requests for resources.Implementations of such techniques are described with respect to FIGS. 8and 9.

FIG. 8 illustrates a flow chart of a process 800 that may alternativelybe used for applications in which it is desirable to track the URLsexplicitly selected by a user, such as tracking popular web pagesexplicitly selected by users or tracking which URLs were selectedexplicitly and thus retrieved intentionally by the user of, for example,a parentally controlled account. In general, in this implementation, acommunication separate from the requests is sent to the proxy server,and includes one or more URLs that correspond to new navigations.

Specifically, browser 200 generates a request (802) and a determinationis made as to whether the request was generated as a result of a newuser navigation (e.g., when the user types a URL into a navigation barof browser 200 or clicks on a hyperlink) (804). If the request is not aresult of a new navigation, then the request is sent to the server(812).

If the request is a result of a new navigation (804), then, optionally,a determination is made as to whether the new navigation is a framenavigation (806). If the new navigation is a frame navigation (806),then the request is sent to the server (812). If the new navigation isnot a frame navigation (804), the URL requested in the request is stored(808) and the request is sent to the server (812). In otherimplementations, action 806 may be omitted such that URLs correspondingto new navigations are stored, regardless of whether they are a framenavigation or not.

The stored URL is then sent to the proxy server in a communicationseparate from the request (812). The URL may be sent once it is stored(that is, it may be sent immediately after it is store, at substantiallythe same time as the request), or the URL may be stored for a period oftime before being sent. Alternatively, or additionally, one or more URLsfor new navigations may be stored, and collectively sent to the proxyserver. In other words, actions 802 through 810 may be performedrepeatedly such that there are multiple, stored URLs. The multiple,stored URLs may then be sent as a group to the proxy server in acommunication separate from a request (or, in another implementation, aspart of a request).

Because the stored URLs are those that correspond to intentionalnavigations of a user, the proxy server can obtain a list of those webpages (or other resources) that the user intentionally requests. Thismay allow for a more accurate portrayal of the web pages requested in aparental controls application or a more accurate portrayal of popularweb pages in a popular webpage application.

As an example of process 800 applied to web page 210, a user enters theURL for webpage 210 (i.e., “www.cnn.com”) into a navigation bar ofbrowser 200. Browser 200 generates a request as a result (802) (whichwill be sent from client computer 111 to, for example, server computer132). This request is the result of a new navigation (804), that is nota frame navigation (806). Consequently, the URL “www.cnn.com” is stored(808). The request is sent to the server computer 132 (810). The requestis first received by proxy server 117. Proxy server 117 then forwardsthe request for webpage 210 to the server computer 132, receives thesource code for webpage 210 from server computer 132, and forwards thesource code for webpage 210 back to client computer 111 so that browser200 can render webpage 210.

An additional communication is sent by browser 200 to proxy server 117,for example, at substantially the same time as the request (812). Thisadditional communication includes the stored URL for webpage 210 (i.e.,www.cnn.com). Proxy server 117 receives the communication and uses theincluded URL, for example, to generate a list of websites visited for aparental control application, or to determine popular websites for apopular website application.

Browser 200 begins processing the source code and rendering webpage 210.As part of rendering webpage 210, browser generates requests for objectsin webpage 210, such as graphics 212 and 216. For example, browser 210begins to render graphic 212 before graphic 216. As a result, browser210 generates a request for graphic 212 (802) (which will be sent fromclient computer 111 to, for example, server computer 131). This requestis not the result of a new navigation (804) because it is beingperformed as part of the rendering. Consequently, the request is sent toserver computer 131 (810). The request is received by proxy server 117,which then forwards the request for graphic 212 to the server computer131, receives graphic 212 from server computer 131, and forwards graphic212 back to client computer 111 so that browser 200 can display graphic212.

Next, browser 210 begins rendering graphic 216. As with graphic 212,browser 210 generates a request for graphic 216 (802) (which will besent to, for example, server computer 133). This request is not theresult of a new navigation (804) because it is being performed as partof the rendering. Consequently, the request is sent to server computer133 (810). Again, the request is first received by proxy server 117,which then forwards the request for graphic 216 to the server computer133, receives graphic 216 from server computer 133, and forwards graphic216 back to client computer 111 so that browser 200 can display graphic216.

Once web page 210 is rendered, if the user, for example, clicks on ahyperlink (e.g., one of the hyperlinks 226), then browser 200 generatesa request (802). This request, however, is the result of a newnavigation (804) and is not a frame navigation (806). Consequently, theURL pointed to by the hyperlink the user clicked on is stored (808) andthe request is sent to the appropriate server computer (810).

An additional communication is sent by browser 200 to proxy server 117,for example, at substantially the same time as the request (812). Thisadditional communication includes the stored URL pointed to by thehyperlink. Proxy server 117 receives the communication and uses theincluded URL, for example, to generate a list of websites visited for aparental control application, or to determine popular websites for apopular website application.

As described above, in other implementations, the stored URLsalternatively may have been sent individually at a time period after thecorresponding request was sent, or may have been sent as a group in acommunication separate from a request (or as part of a request).

FIG. 9 illustrates a flow chart of another process 900 that mayalternatively be used for applications in which it is desirable to trackthe URLs explicitly selected by a user, such as tracking popular webpages or tracking which URLs were explicitly selected and thus retrievedintentionally by, for example, the user of a parentally controlledaccount. In general, in this implementation, a header is appended orotherwise included in the requests that correspond to new navigations,or new navigations that are not frame navigations, but not included inother requests. The header indicates that the request corresponds to anew navigation, or a new navigation that is not a frame navigation. Theheader may have the same form as the page view header described above,or may have a different form that simply indicates that the URL in therequest is the result of a new navigation, or a new navigation that isnot a frame navigation. This header may be referred to as a navigationheader.

Specifically, browser 200 generates a request (902) and a determinationis made as to whether the request was generated as a result of a newuser navigation (e.g., when the user types a URL into a navigation barof browser 200 or clicks on a hyperlink) (904). If the request is not aresult of a new navigation, then the request is sent to the server(910).

If the request is a result of a new navigation (904), then, optionally,a determination is made as to whether the new navigation is a framenavigation (906). If the new navigation is a frame navigation (906),then the request is sent to the server (910). If the new navigation is aframe navigation (904), then a navigation header is appended orotherwise included in the request (908) and the request is sent to theserver (910). The navigation header indicates that the URL in therequest is the result of a new navigation that is not a framenavigation. In other implementations, action 906 may be omitted suchthat the navigation header is appended to requests corresponding to newnavigations, regardless of whether the navigations are frame navigationsor not.

Through a process such as process 900, the requests for URLsintentionally navigated to by a user include a navigation headerindicating that the request is a result of a user navigation.Accordingly, the proxy server can store a list of those web pages (orother resources) that the user intentionally requests. This may allowfor a more accurate portrayal of the web pages requested in a parentalcontrols application or a more accurate portrayal of popular web pagesin a popular webpage application.

The techniques described above are not limited to any particularhardware or software configuration. Rather, they may be implementedusing hardware, software, or a combination of both. The methods andprocesses described may be implemented as computer programs that areexecuted on programmable computers comprising at least one processor andat least one data storage system. The programs may be implemented in ahigh-level programming language and may also be implemented in assemblyor other lower level languages, if desired.

Any such program will typically be stored on a computer-usable storagemedium or device (e.g., CD-Rom, RAM, or magnetic disk). When read intothe processor of the computer and executed, the instructions of theprogram cause the programmable computer to carry out the variousoperations described above.

A number of implementations have been described. Nevertheless, it willbe understood that various modifications may be made. For example, whilethe processing based on the page view field has been described as beingperformed at a proxy server, the processing may be performed at otherlocations, such as the server computer that includes the requestedobject. Also, while preventing the reporting of certain URLS has beendescribed as not storing the URLs depending on the page view field, analternative implementation may include storing the request URLs and theURLs in the page view field, and then filtering the requested URLs bythe stored URLs from the page view field such URLs for objects in a webpage or other URLs not intentionally requested by the user are notreported. Also, while appending has been described as being used toincorporate the page view header, other manners may be used to includeit in a request. In addition, while URLs have been described as beingused as identifiers, other types of identifiers of webpages or otherresources may be used. Accordingly, other implementations are within thescope of the following claims.

What is claimed is:
 1. A method comprising the following operationsperformed by at least one processor: receiving, by a first servercomputer from a client, a request for a resource to be rendered in a webpage, the request including an identification of the resource;determining, by the first server computer, that the request includes apage view header and, based on the page view header, an identificationof the web page; determining, based on the page view header, whether auser has explicitly requested the resource; when it is determined thatthe user has explicitly requested the resource: storing theidentification of the web page and updating a list of web pagesexplicitly requested by the user; determining, at the first servercomputer, based on the identification of the web page, whether an accessto the requested resource by the user is permitted by: comparing theidentification of the web page with a list of identification of excludedweb sites; and determining whether the identification of the web page isin the list; and permitting access to the requested resource when it isdetermined that the user is permitted access; and when it is determinedthat the user has not explicitly requested the resource: sending therequest without the page view header to a second server computer thathas the requested resource.
 2. The method of claim 1, wherein permittingaccess to the requested resource further comprises: removing the pageview header from the request; and sending the request without the pageview header to a second server computer that has the requested resource.3. The method of claim 1, wherein the page view header further indicatesthat the resource is requested as a result of rendering the web page. 4.The method of claim 1, further comprising: determining whether theidentification of the web page has been stored in the first servercomputer.
 5. The method of claim 1, wherein determining whether anaccess to the requested resource by the user is permitted furthercomprises: comparing the identification of the web page with a list ofidentifications of allowed web sites; determining that theidentification of the web page is in the list; and sending the requestwithout the page view header to a second server computer that has therequested resource.
 6. The method of claim 5, further comprising sendinga response to the client indicating that the access to the resource bythe user is not permitted.
 7. The method of claim 1, further comprising:storing, by the first server computer, a relationship between the webpage and the resource; and determining whether an access to the resourceby the user is permitted based on the stored relationship.
 8. The methodof claim 1, wherein the request is an HTTP request.
 9. A non-transitorycomputer-useable storage medium storing a computer program, the computerprogram comprising instructions for causing a computer to: receive, froma client, a request for a resource to be rendered in a web page, therequest including an identification of the resource; determine, that therequest includes a page view header and, based on the page view header,an identification of the web page; determine, based on the page viewheader, whether a user has explicitly requested the resource; when it isdetermined that the user has explicitly requested the resource: storethe identification of the web page and update a list of web pagesexplicitly requested by the user; determine, by the first servercomputer, based on the identification of the web page, whether an accessto the requested resource by the user is permitted, by: comparing theidentification of the web page with a list of identification of excludedweb sites; and determining whether the identification of the web page isin the list; and permit access to the requested resource when it isdetermined that the user is permitted access; and when it is determinedthat the user has not explicitly requested the resource: send therequest without the page view header to a server computer that has therequested resource.
 10. The non-transitory computer-useable storagemedium of claim 9, wherein the determination whether an access to therequested resource by the user is permitted further comprisesinstructions for causing a computer to: compare the identification ofthe web page with a list of identifications of allowed web sites;determine that the identification of the web page is in the list; andsend the request without the page view header to a server computer thathas the requested resource.
 11. The non-transitory computer-useablestorage medium of claim 10, wherein the computer program furthercomprising instructions for causing a computer to send a response to theclient indicating that the access to the resource by the user is notpermitted.
 12. The non-transitory computer-useable storage medium ofclaim 9, wherein the computer program further comprising instructionsfor causing a computer to: store a relationship between the web page andthe resource; and determine whether an access to the resource by theuser is permitted based on the stored relationship.
 13. Thenon-transitory computer-useable storage medium of claim 9, whereinpermitting access to the requested resource further comprisesinstructions for causing a computer to: remove the page view header fromthe request; and send the request without the page view header to aserver computer that has the requested resource.
 14. The non-transitorycomputer-useable storage medium of claim 9, wherein the page view headerfurther indicates that the resource is requested as a result ofrendering the web page.
 15. The non-transitory computer-useable storagemedium of claim 9, further comprising instructions for causing acomputer to: determine whether the identification of the web page hasbeen stored.